{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ECSE415 - Intro to Computer Vision\n",
    "## Tutorial 9 - Introduction to Machine Learning  using Scikit-Learn\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A machine learning problem consists of $n$ instances of data and attempts to predict properties of unseen data.\n",
    "\n",
    "Learning problems can be separated into several categories:\n",
    "* Supervised Learning: data comes with additional attributes that we want to predict. This problem can be either:\n",
    " * Classification: Samples belong to two or more classes and the algorithm learns from already labeled data to         predict to class of unseen data.\n",
    " * Regression: the desired output consists of one or more continuous variables.\n",
    "* Unsupervised Learning: data consists of a set of input vectors without any corresponding target values. Examples include clustering, density estimation, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input data sample:\n",
      "[[  0.   0.   5.  13.   9.   1.   0.   0.   0.   0.  13.  15.  10.  15.\n",
      "    5.   0.   0.   3.  15.   2.   0.  11.   8.   0.   0.   4.  12.   0.\n",
      "    0.   8.   8.   0.   0.   5.   8.   0.   0.   9.   8.   0.   0.   4.\n",
      "   11.   0.   1.  12.   7.   0.   0.   2.  14.   5.  10.  12.   0.   0.\n",
      "    0.   0.   6.  13.  10.   0.   0.   0.]\n",
      " [  0.   0.   0.  12.  13.   5.   0.   0.   0.   0.   0.  11.  16.   9.\n",
      "    0.   0.   0.   0.   3.  15.  16.   6.   0.   0.   0.   7.  15.  16.\n",
      "   16.   2.   0.   0.   0.   0.   1.  16.  16.   3.   0.   0.   0.   0.\n",
      "    1.  16.  16.   6.   0.   0.   0.   0.   1.  16.  16.   6.   0.   0.\n",
      "    0.   0.   0.  11.  16.  10.   0.   0.]\n",
      " [  0.   0.   0.   4.  15.  12.   0.   0.   0.   0.   3.  16.  15.  14.\n",
      "    0.   0.   0.   0.   8.  13.   8.  16.   0.   0.   0.   0.   1.   6.\n",
      "   15.  11.   0.   0.   0.   1.   8.  13.  15.   1.   0.   0.   0.   9.\n",
      "   16.  16.   5.   0.   0.   0.   0.   3.  13.  16.  16.  11.   5.   0.\n",
      "    0.   0.   0.   3.  11.  16.   9.   0.]]\n",
      "Input data dimensions: (1797, 64)\n",
      "Output labels:\n",
      "[0 1 2]\n",
      "Output data dimensions: (1797,)\n"
     ]
    }
   ],
   "source": [
    "from sklearn import datasets\n",
    "\n",
    "digits = datasets.load_digits()\n",
    "print(\"Input data sample:\")\n",
    "print(digits.data[0:3])\n",
    "print(\"Input data dimensions:\", digits.data.shape)\n",
    "print(\"Output labels:\")\n",
    "print(digits.target[0:3])\n",
    "print(\"Output data dimensions:\", digits.target.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The task is to predict the output label (number from 0-9) for each input image. Since we are provided with examples of each of the 10 possible classes, we can now attempt to *fit* an estimator to be able to *predict* the classes to which unseen samples belong.\n",
    "\n",
    "We can begin by looking at a Support Vector Machine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted Label: [0 8 9 8]\n",
      "Actual Label: [0 8 9 8]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkYAAAJCCAYAAAAlTAh6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X+s5Xd5J/b3w4wdwBgPBRaBjXy9DbU2QmIczXoVkSWz\nuE7NhvCjWu3aEihGWU20WlKbpGIhVWuzqqr2j6Ch0jbt1GA7wtgCg12CCAiEHRY1/BibYcE/yII9\nlschDJSMwYTi2jz9Yz5TXejAPcf3nPO9vvf1kq7m3nu+c97P9Ywfv/39nh/V3QEAIHnG1AMAAGwV\nihEAwKAYAQAMihEAwKAYAQAMihEAwKAYAQAMihEAwKAYAQAMu5dxp1W1I19O+/nPf/5k2Wtra5Nl\n/+hHP5os+8c//vFk2UePHp0s+8knn5wsu7trsvAV2Kn7a9euXZNlT7m/zj777Mmyv/rVr06WPeUO\nmdIs+2spxWin+u3f/u3Jsq+//vrJsr/yla9Mlj1lObnyyisnyz5x4sRk2WxPUxaEP/7jP54se//+\n/ZNlT1kI7ZCfz6U0AIBBMQIAGBQjAIBBMQIAGBQjAIBBMQIAGBQjAIBBMQIAGBQjAIBBMQIAGGYq\nRlV1WVV9vaq+UVXvWPZQAItifwHz2LAYVdWuJP8uyWuS/EqSK6rqV5Y9GMBm2V/AvGY5Y3Rxkm90\n9wPd/XiSW5K8frljASyE/QXMZZZidG6Sh9d9fWx8D2Crs7+Auexe1B1V1YEkBxZ1fwCrYn8Bp8xS\njB5J8tJ1X583vvdTuvtQkkNJUlW9kOkANsf+AuYyy6W0LyV5WVVdUFVnJrk8yUeXOxbAQthfwFw2\nPGPU3U9U1VuTfDLJriTv6+57lj4ZwCbZX8C8ZnqMUXd/PMnHlzwLwMLZX8A8vPI1AMCgGAEADIoR\nAMCgGAEADIoRAMCgGAEADIoRAMCgGAEADIoRAMAw0ytfM5vrr79+suxHH310suwTJ05Mlv36179+\nsmzYTu68887Jsl/xildMlv2ud71rsuw9e/ZMlj2lKf+bMQtnjAAABsUIAGBQjAAABsUIAGBQjAAA\nBsUIAGBQjAAABsUIAGBQjAAABsUIAGBQjAAABsUIAGDYsBhV1fuq6nhVfW0VAwEskh0GzGOWM0Y3\nJLlsyXMALMsNscOAGW1YjLr7s0m+t4JZABbODgPm4TFGAADD7kXdUVUdSHJgUfcHsCr2F3DKwopR\ndx9KcihJqqoXdb8Ay2Z/Aae4lAYAMMzydP2bk/xlkgur6lhV/e7yxwJYDDsMmMeGl9K6+4pVDAKw\nDHYYMA+X0gAABsUIAGBQjAAABsUIAGBQjAAABsUIAGBQjAAABsUIAGBQjAAAhoW9iexWsXfv3qlH\nmMS11147WfbBgwcnyz5y5Mhk2W94wxsmy77hhhsmy2Z7esUrXjFZ9qOPPjpZ9tra2mTZDz744GTZ\nb3zjGyfLvv322yfLnoUzRgAAg2IEADAoRgAAg2IEADAoRgAAg2IEADAoRgAAg2IEADAoRgAAg2IE\nADAoRgAAg2IEADBsWIyq6qVVdUdV3VtV91TVVasYDGCz7C9gXrtnOOaJJH/Y3XdX1dlJ7qqqT3X3\nvUueDWCz7C9gLhueMerub3X33ePzHyS5L8m5yx4MYLPsL2Bes5wx+v9U1VqSi5J84TS3HUhyYCFT\nASyY/QXMYuZiVFXPSfLhJFd39/d/9vbuPpTk0Di2FzYhwCbZX8CsZnpWWlWdkZNL5abu/shyRwJY\nHPsLmMcsz0qrJO9Ncl93v3v5IwEshv0FzGuWM0avTPLmJK+uqiPj458ueS6ARbC/gLls+Bij7v5c\nklrBLAALZX8B8/LK1wAAg2IEADAoRgAAg2IEADAoRgAAg2IEADAoRgAAg2IEADAoRgAAw4avfP10\ns2fPnqlHmMTBgwenHmESR44cmSx7bW1tsmxYtHe9612TZV9zzTWTZf/O7/zOZNlve9vbJsu+/fbb\nJ8ve6pwxAgAYFCMAgEExAgAYFCMAgEExAgAYFCMAgEExAgAYFCMAgEExAgAYFCMAgEExAgAYFCMA\ngGHDYlRVz6yqL1bVV6rqnqqa7p0GAeZgfwHz2j3DMT9O8urufqyqzkjyuar68+7+/JJnA9gs+wuY\ny4bFqLs7yWPjyzPGRy9zKIBFsL+Aec30GKOq2lVVR5IcT/Kp7v7CaY45UFWHq+rwoocEeKrsL2Ae\nMxWj7n6yu/cmOS/JxVX18tMcc6i793X3vkUPCfBU2V/APOZ6Vlp3n0hyR5LLljMOwHLYX8AsZnlW\n2guras/4/FlJLk1y/7IHA9gs+wuY1yzPSntxkhuraldOFqkPdvfHljsWwELYX8BcZnlW2n9IctEK\nZgFYKPsLmJdXvgYAGBQjAIBBMQIAGBQjAIBBMQIAGBQjAIBBMQIAGBQjAIBBMQIAGGZ5S5Cnlf37\n9089Ais05Z/3tddeO1k2LNqePXumHmHHOXLkyNQjcBrOGAEADIoRAMCgGAEADIoRAMCgGAEADIoR\nAMCgGAEADIoRAMCgGAEADIoRAMCgGAEADIoRAMAwczGqql1V9eWq+tgyBwJYNPsLmNU8Z4yuSnLf\nsgYBWCL7C5jJTMWoqs5L8ltJrlvuOACLZX8B85j1jNHBJG9P8pOfd0BVHaiqw1V1eCGTASyG/QXM\nbMNiVFWvTXK8u+/6Rcd196Hu3tfd+xY2HcAm2F/AvGY5Y/TKJK+rqqNJbkny6qp6/1KnAlgM+wuY\ny4bFqLvf2d3ndfdaksuTfKa737T0yQA2yf4C5uV1jAAAht3zHNzddya5cymTACyR/QXMwhkjAIBB\nMQIAGBQjAIBBMQIAGBQjAIBBMQIAGBQjAIBBMQIAGBQjAIBhrle+fjo4cuTI1CNM4uqrr54se21t\nbbLs888/f7LsEydOTJYNi3bVVVdNlv3QQw9Nlj3lDrn99tsny96zZ89k2VudM0YAAINiBAAwKEYA\nAINiBAAwKEYAAINiBAAwKEYAAINiBAAwKEYAAINiBAAwKEYAAINiBAAwzPQmslV1NMkPkjyZ5Inu\n3rfMoQAWxf4C5jFTMRr+SXd/d2mTACyP/QXMxKU0AIBh1mLUST5dVXdV1YHTHVBVB6rqcFUdXtx4\nAJtmfwEzm/VS2q939yNV9feSfKqq7u/uz64/oLsPJTmUJFXVC54T4Kmyv4CZzXTGqLsfGb8eT3Jb\nkouXORTAothfwDw2LEZVdVZVnX3q8yS/meRryx4MYLPsL2Bes1xKe1GS26rq1PEf6O5PLHUqgMWw\nv4C5bFiMuvuBJK9YwSwAC2V/AfPydH0AgEExAgAYFCMAgEExAgAYFCMAgEExAgAYFCMAgEExAgAY\nFCMAgKG6F/9G0lO+O/WePXumis7Ro0cnyz7nnHMmy96pnve8502WfeLEicmyu7smC1+BKffXlKb8\nO7VTTbm3x9vk7Diz7C9njAAABsUIAGBQjAAABsUIAGBQjAAABsUIAGBQjAAABsUIAGBQjAAABsUI\nAGBQjAAAhpmKUVXtqapbq+r+qrqvqn5t2YMBLIL9Bcxj94zHvSfJJ7r7n1XVmUmevcSZABbJ/gJm\ntmExqqpzkrwqyZVJ0t2PJ3l8uWMBbJ79BcxrlktpFyT5TpLrq+rLVXVdVZ215LkAFsH+AuYySzHa\nneRXk/xJd1+U5IdJ3vGzB1XVgao6XFWHFzwjwFNlfwFzmaUYHUtyrLu/ML6+NScXzU/p7kPdva+7\n9y1yQIBNsL+AuWxYjLr7b5I8XFUXjm9dkuTepU4FsAD2FzCvWZ+V9vtJbhrP6HggyVuWNxLAQtlf\nwMxmKkbdfSSJU8zA0479BczDK18DAAyKEQDAoBgBAAyKEQDAoBgBAAyKEQDAoBgBAAyKEQDAoBgB\nAAyKEQDAMOt7pT1tnDhxYrLsq6++erLs66+/frLshx56aLLsa6+9drLsKf+uwaLt3bt3suwpd+f+\n/fsnyz548OBk2fx8zhgBAAyKEQDAoBgBAAyKEQDAoBgBAAyKEQDAoBgBAAyKEQDAoBgBAAyKEQDA\noBgBAAwbFqOqurCqjqz7+H5VTffGNgAzsr+AeW34JrLd/fUke5OkqnYleSTJbUueC2DT7C9gXvNe\nSrskyTe7e7q3Uwd4auwvYEPzFqPLk9y8jEEAlsz+AjY0czGqqjOTvC7Jh37O7Qeq6nBVHV7UcACL\nYH8Bs9rwMUbrvCbJ3d397dPd2N2HkhxKkqrqBcwGsCj2FzCTeS6lXRGnoYGnJ/sLmMlMxaiqzkpy\naZKPLHccgMWyv4B5zHQprbt/mOT5S54FYOHsL2AeXvkaAGBQjAAABsUIAGBQjAAABsUIAGBQjAAA\nBsUIAGBQjAAABsUIAGBQjAAAhupe/BtJV9V3kjz0FH/7C5J8d4HjyJa9lXKf7tnnd/cLFzXMVmR/\nyZa9bbNn2l9LKUabUVWHu3ufbNnbMXcnZ+8EO/XPVrbs7ZTtUhoAwKAYAQAMW7EYHZItexvn7uTs\nnWCn/tnKlr1tsrfcY4wAAKayFc8YAQBMQjECABi2TDGqqsuq6utV9Y2qeseKs99XVcer6msrzn1p\nVd1RVfdW1T1VddUKs59ZVV+sqq+M7HetKnvdDLuq6stV9bEV5x6tqq9W1ZGqOrzi7D1VdWtV3V9V\n91XVr60o98Lx8576+H5VXb2K7J1iqh021f4a2Tt2h9lf23d/bYnHGFXVriR/leTSJMeSfCnJFd19\n74ryX5XksSR/2t0vX0XmyH1xkhd3991VdXaSu5K8YRU/d1VVkrO6+7GqOiPJ55Jc1d2fX3b2uhn+\nIMm+JM/t7teuMPdokn3dvfIXKKuqG5P8++6+rqrOTPLs7j6x4hl2JXkkyT/q7qf6QoasM+UOm2p/\njewdu8Psr+27v7bKGaOLk3yjux/o7seT3JLk9asK7+7PJvneqvLW5X6ru+8en/8gyX1Jzl1Rdnf3\nY+PLM8bHylpyVZ2X5LeSXLeqzKlV1TlJXpXkvUnS3Y+veqkMlyT5plK0UJPtsKn218jekTvM/tre\n+2urFKNzkzy87utjWdG/XFtFVa0luSjJF1aYuauqjiQ5nuRT3b2y7CQHk7w9yU9WmHlKJ/l0Vd1V\nVQdWmHtBku8kuX6cgr+uqs5aYf4plye5eYLc7cwO21k7zP7axvtrqxSjHa2qnpPkw0mu7u7vryq3\nu5/s7r1JzktycVWt5DR8Vb02yfHuvmsVeafx6+Pnfk2Sfz0uRazC7iS/muRPuvuiJD9MsurH052Z\n5HVJPrTKXLa3nbTD7K/tv7+2SjF6JMlL13193vjetjeujX84yU3d/ZEpZhinQ+9IctmKIl+Z5HXj\nWvktSV5dVe9fUXa6+5Hx6/Ekt+XkZZBVOJbk2Lr/q701JxfNKr0myd3d/e0V5253dtjO2WH210nb\ndn9tlWL0pSQvq6oLRiO8PMlHJ55p6caDB9+b5L7ufveKs19YVXvG58/KyQeN3r+K7O5+Z3ef191r\nOfln/ZnuftMqsqvqrPEg0YzTwL+ZZCXP5unuv0nycFVdOL51SZKVPMFgnSviMtoy2GE7ZIfZX9t/\nf+1edsAsuvuJqnprkk8m2ZXkfd19z6ryq+rmJPuTvKCqjiW5prvfu4LoVyZ5c5KvjuvkSfJH3f3x\nFWS/OMmN4xH+z0jywe5e6dNOJ/KiJLed3OfZneQD3f2JFeb/fpKbxn88H0jyllUFj0V6aZLfW1Xm\nTjHlDptwfyV22KrZXyvYX1vi6foAAFvBVrmUBgAwOcUIAGBQjAAABsUIAGBQjAAABsUIAGBQjAAA\nBsUIAGBQjAAABsUIAGBQjAAABsUIAGBQjAAABsUIAGBQjAAABsUIAGBQjAAABsUIgC2hqv7Xqvpv\nF30szKO6e+oZANjmqupokhcleSLJk0nuTfKnSQ519082ed/7k7y/u89b5e9le3LGCIBV+e3uPjvJ\n+Un+xyT/Jsl7px0JfppiBMBKdfej3f3RJP8iye9U1cuTpKpuqKr//tRxVfX2qvpWVf11Vf3Lquqq\n+uX1x1bVWUn+PMlLquqx8fGSqrq4qg5X1fer6ttV9e5ZZquqO8f9/p/jvv6sqp5fVTeN+/pSVa2t\nO/49VfXwuO2uqvrH6257VlXdWFV/W1X3jZ/n2LrbX1JVH66q71TVg1X1X23unyyLoBgBMInu/mKS\nY0n+8c/eVlWXJfmDJP95kl9Osv/n3McPk7wmyV9393PGx18neU+S93T3c5P8p0k+OMdolyd5c5Jz\nx+/9yyTXJ/lPktyX5Jp1x34pyd5x2weSfKiqnjluuybJWpK/n+TSJG9a9/M9I8mfJfnKyLkkydVV\n9V/MMSdLoBgBMKW/zslS8bP+eZLru/ue7v67JNfOeb//T5JfrqoXdPdj3f35OX7v9d39ze5+NCfP\nRn2zuz/d3U8k+VCSi04d2N3v7+7/q7uf6O4/TvJLSS5c9zP8D939t919LMn/vC7jHyZ5YXf/2+5+\nvLsfSPK/52QpY0KKEQBTOjfJ907z/ZckeXjd1w+f5phf5HeT/GdJ7h+Xv147x+/99rrPf3Sar59z\n6ouq+q/HZbJHq+pEknOSvGDc/It+hvNz8vLfiVMfSf4oJx+gzoR2Tz0AADtTVf3DnCxGnzvNzd9K\nsv6ZYi/9BXf1/3t6dXf/xyRXjEtW/2WSW6vq+ePS20KMxxO9PScvg93T3T+pqr9NUuOQUz/Dvaf5\nGR5O8mB3v2xR87AYzhgBsFJV9dxxBueWnHyq/FdPc9gHk7ylqv5BVT07yS96zaJvJ3l+VZ2zLuNN\nVfXC8VIAJ8a3N/WyAKdxdk6+/MB3kuyuqv8uyXPX3f7BJO+squdV1blJ3rruti8m+UFV/ZvxIO1d\nVfXyURaZkGIEwKr8WVX9ICfPlvw3Sd6d5C2nO7C7/zwnH5NzR5JvJDn1GKEfn+bY+5PcnOSBcVnq\nJUkuS3JPVT2Wkw/Evry7f7Tgn+eTST6R5K+SPJTk/85PXy77tzn54PIHk3w6ya2n5u/uJ5O8Nicf\nuP1gku8muS4nL8UxIS/wCMCWV1X/IMnXkvzSeBD0005V/aucLGi/MfUs/HzOGAGwJVXVG6vql6rq\neUn+pyR/9nQqRVX14qp6ZVU9o6ouTPKHSW6bei5+McUIgK3q95IcT/LNnHwbkX817ThzOzPJ/5bk\nB0k+k+T/SPK/TDoRG3IpDQBgcMYIAGBQjAAAhqW8wGNV7cjrc2eeeeZk2S95yUsmy/67v/u7ybKP\nHz8+WfZO1d218VFPXzt1f62trU2WffbZZ0+W/d3vfney7Cn315NPPjlZ9pRm2V9LeYyRxbJ61157\n7WTZR44cmSz74MGDk2XvVIrR9nTDDTdMlr1///7Jsqf8uafcXydOnNj4oG1olv3lUhoAwKAYAQAM\nihEAwKAYAQAMihEAwKAYAQAMihEAwKAYAQAMihEAwKAYAQAMMxWjqrqsqr5eVd+oqncseyiARbG/\ngHlsWIyqaleSf5fkNUl+JckVVfUryx4MYLPsL2Bes5wxujjJN7r7ge5+PMktSV6/3LEAFsL+AuYy\nSzE6N8nD674+Nr73U6rqQFUdrqrDixoOYJPsL2Auuxd1R919KMmhJKmqXtT9Aiyb/QWcMssZo0eS\nvHTd1+eN7wFsdfYXMJdZitGXkrysqi6oqjOTXJ7ko8sdC2Ah7C9gLhteSuvuJ6rqrUk+mWRXkvd1\n9z1Lnwxgk+wvYF4zPcaouz+e5ONLngVg4ewvYB5e+RoAYFCMAAAGxQgAYFCMAAAGxQgAYFCMAAAG\nxQgAYFCMAAAGxQgAYKjuxb+R9E59d+qjR49Oln3++edPlj2lRx99dLLstbW1ybJPnDgxWXZ312Th\nKzDl/pry79SDDz44WfZDDz00Wfa11147Wfbtt98+WfaUO2RKs+wvZ4wAAAbFCABgUIwAAAbFCABg\nUIwAAAbFCABgUIwAAAbFCABgUIwAAAbFCABgUIwAAAbFCABg2LAYVdX7qup4VX1tFQMBLJIdBsxj\nljNGNyS5bMlzACzLDbHDgBltWIy6+7NJvreCWQAWzg4D5rF7UXdUVQeSHFjU/QGsiv0FnLKwYtTd\nh5IcSpKq6kXdL8Cy2V/AKZ6VBgAwKEYAAMMsT9e/OclfJrmwqo5V1e8ufyyAxbDDgHls+Bij7r5i\nFYMALIMdBszDpTQAgEExAgAYFCMAgEExAgAYFCMAgEExAgAYFCMAgEExAgAYFCMAgGHDV75+utm/\nf/9k2eeff/5k2W9729smy77zzjsny/7yl788WfaVV145WfbBgwcny2Z5jh49Oln2o48+Oln2nj17\nJsteW1ubLHvKP+8p/5lvdc4YAQAMihEAwKAYAQAMihEAwKAYAQAMihEAwKAYAQAMihEAwKAYAQAM\nihEAwKAYAQAMGxajqnppVd1RVfdW1T1VddUqBgPYLPsLmNcsbyL7RJI/7O67q+rsJHdV1ae6+94l\nzwawWfYXMJcNzxh197e6++7x+Q+S3Jfk3GUPBrBZ9hcwr7keY1RVa0kuSvKFZQwDsCz2FzCLWS6l\nJUmq6jlJPpzk6u7+/mluP5DkwAJnA1gI+wuY1UzFqKrOyMmlclN3f+R0x3T3oSSHxvG9sAkBNsH+\nAuYxy7PSKsl7k9zX3e9e/kgAi2F/AfOa5TFGr0zy5iSvrqoj4+OfLnkugEWwv4C5bHgprbs/l6RW\nMAvAQtlfwLy88jUAwKAYAQAMihEAwKAYAQAMihEAwKAYAQAMihEAwKAYAQAMihEAwKAYAQAMG74l\nyNPNnj17ph5hEnv37p16hB3nyJEjU48AC3PllVdOln3bbbdNln3NNddMln3jjTdOls3P54wRAMCg\nGAEADIoRAMCgGAEADIoRAMCgGAEADIoRAMCgGAEADIoRAMCgGAEADIoRAMCwYTGqqmdW1Rer6itV\ndU9VvWsVgwFslv0FzGuWN5H9cZJXd/djVXVGks9V1Z939+eXPBvAZtlfwFw2LEbd3UkeG1+eMT56\nmUMBLIL9BcxrpscYVdWuqjqS5HiST3X3F5Y7FsBi2F/APGYqRt39ZHfvTXJekour6uU/e0xVHaiq\nw1V1eNFDAjxV9hcwj7meldbdJ5LckeSy09x2qLv3dfe+RQ0HsCj2FzCLWZ6V9sKq2jM+f1aSS5Pc\nv+zBADbL/gLmNcuz0l6c5Maq2pWTReqD3f2x5Y4FsBD2FzCXWZ6V9h+SXLSCWQAWyv4C5uWVrwEA\nBsUIAGBQjAAABsUIAGBQjAAABsUIAGBQjAAABsUIAGBQjAAABsUIAGCY5b3SnlZuv/32ybLf+MY3\nTpZ98ODBybL3798/WTawGFdfffVk2Y8++uhk2VNaW1ubegROwxkjAIBBMQIAGBQjAIBBMQIAGBQj\nAIBBMQIAGBQjAIBBMQIAGBQjAIBBMQIAGBQjAIBh5mJUVbuq6stV9bFlDgSwaPYXMKt5zhhdleS+\nZQ0CsET2FzCTmYpRVZ2X5LeSXLfccQAWy/4C5jHrGaODSd6e5CdLnAVgGewvYGYbFqOqem2S4919\n1wbHHaiqw1V1eGHTAWyC/QXMa5YzRq9M8rqqOprkliSvrqr3/+xB3X2ou/d1974FzwjwVNlfwFw2\nLEbd/c7uPq+715JcnuQz3f2mpU8GsEn2FzAvr2MEADDsnufg7r4zyZ1LmQRgiewvYBbOGAEADIoR\nAMCgGAEADIoRAMCgGAEADIoRAMCgGAEADIoRAMCgGAEADIoRAMBQ3b34O61a/J3CaSzj7++sLrjg\ngsmyjx49Oll2d9dk4Ssw5f7av3//VNG54447Jst+y1veMln2lP8u7dR/5jfccMNk2bPsL2eMAAAG\nxQgAYFCMAAAGxQgAYFCMAAAGxQgAYFCMAAAGxQgAYFCMAAAGxQgAYFCMAACG3bMcVFVHk/wgyZNJ\nnujufcscCmBR7C9gHjMVo+GfdPd3lzYJwPLYX8BMXEoDABhmLUad5NNVdVdVHVjmQAALZn8BM5v1\nUtqvd/cjVfX3knyqqu7v7s+uP2AsHEsH2GrsL2BmM50x6u5Hxq/Hk9yW5OLTHHOou/d5YCOwldhf\nwDw2LEZVdVZVnX3q8yS/meRryx4MYLPsL2Bes1xKe1GS26rq1PEf6O5PLHUqgMWwv4C5bFiMuvuB\nJK9YwSwAC2V/AfPydH0AgEExAgAYFCMAgEExAgAYFCMAgEExAgAYFCMAgEExAgAYFCMAgEExAgAY\nZnmvNGa0f//+ybL37t07WTawGFPukClN+XMfPXp0suwpra2tTT3CluWMEQDAoBgBAAyKEQDAoBgB\nAAyKEQDAoBgBAAyKEQDAoBgBAAyKEQDAoBgBAAyKEQDAMFMxqqo9VXVrVd1fVfdV1a8tezCARbC/\ngHnM+iay70nyie7+Z1V1ZpJnL3EmgEWyv4CZbViMquqcJK9KcmWSdPfjSR5f7lgAm2d/AfOa5VLa\nBUm+k+T6qvpyVV1XVWf97EFVdaCqDlfV4YVPCfDU2F/AXGYpRruT/GqSP+nui5L8MMk7fvag7j7U\n3fu6e9+CZwR4quwvYC6zFKNjSY519xfG17fm5KIB2OrsL2AuGxaj7v6bJA9X1YXjW5ckuXepUwEs\ngP0FzGvWZ6X9fpKbxjM6HkjyluWNBLBQ9hcws5mKUXcfSeLaO/C0Y38B8/DK1wAAg2IEADAoRgAA\ng2IEADAoRgAAg2IEADAoRgAAg2IEADAoRgAAg2IEADDM+l5pzGDPnj2TZb/hDW+YLPs3fuM3Jsv+\ni7/4i8myjx49Olk229PBgwenHmES+/fv35HZU+6vnfp3bRbOGAEADIoRAMCgGAEADIoRAMCgGAEA\nDIoRAMCgGAEADIoRAMCgGAEADIoRAMCgGAEADBsWo6q6sKqOrPv4flVdvYrhADbD/gLmteGbyHb3\n15PsTZKq2pXkkSS3LXkugE2zv4B5zXsp7ZIk3+zuh5YxDMAS2V/AhjY8Y/QzLk9y8+luqKoDSQ5s\neiKA5bC/gA3NfMaoqs5M8rokHzrd7d19qLv3dfe+RQ0HsAj2FzCreS6lvSbJ3d397WUNA7Ak9hcw\nk3mK0RVHHhSKAAAEjElEQVT5OaehAbY4+wuYyUzFqKrOSnJpko8sdxyAxbK/gHnM9ODr7v5hkucv\neRaAhbO/gHl45WsAgEExAgAYFCMAgEExAgAYFCMAgEExAgAYFCMAgEExAgAYFCMAgEExAgAYqrsX\nf6dV30ny0FP87S9I8t0FjiNb9lbKfbpnn9/dL1zUMFuR/SVb9rbNnml/LaUYbUZVHe7ufbJlb8fc\nnZy9E+zUP1vZsrdTtktpAACDYgQAMGzFYnRItuxtnLuTs3eCnfpnK1v2tsneco8xAgCYylY8YwQA\nMIktU4yq6rKq+npVfaOq3rHi7PdV1fGq+tqKc19aVXdU1b1VdU9VXbXC7GdW1Rer6isj+12ryl43\nw66q+nJVfWzFuUer6qtVdaSqDq84e09V3VpV91fVfVX1ayvKvXD8vKc+vl9VV68ie6eYaodNtb9G\n9o7dYfbX9t1fW+JSWlXtSvJXSS5NcizJl5Jc0d33rij/VUkeS/Kn3f3yVWSO3BcneXF3311VZye5\nK8kbVvFzV1UlOau7H6uqM5J8LslV3f35ZWevm+EPkuxL8tzufu0Kc48m2dfdK38djqq6Mcm/7+7r\nqurMJM/u7hMrnmFXkkeS/KPufqqv18M6U+6wqfbXyN6xO8z+2r77a6ucMbo4yTe6+4HufjzJLUle\nv6rw7v5sku+tKm9d7re6++7x+Q+S3Jfk3BVld3c/Nr48Y3ysrCVX1XlJfivJdavKnFpVnZPkVUne\nmyTd/fiql8pwSZJvKkULNdkOm2p/jewducPsr+29v7ZKMTo3ycPrvj6WFf3LtVVU1VqSi5J8YYWZ\nu6rqSJLjST7V3SvLTnIwyduT/GSFmad0kk9X1V1VdWCFuRck+U6S68cp+Ouq6qwV5p9yeZKbJ8jd\nzuywnbXD7K9tvL+2SjHa0arqOUk+nOTq7v7+qnK7+8nu3pvkvCQXV9VKTsNX1WuTHO/uu1aRdxq/\nPn7u1yT51+NSxCrsTvKrSf6kuy9K8sMkq3483ZlJXpfkQ6vMZXvbSTvM/tr++2urFKNHkrx03dfn\nje9te+Pa+IeT3NTdH5lihnE69I4kl60o8pVJXjeuld+S5NVV9f4VZae7Hxm/Hk9yW05eBlmFY0mO\nrfu/2ltzctGs0muS3N3d315x7nZnh+2cHWZ/nbRt99dWKUZfSvKyqrpgNMLLk3x04pmWbjx48L1J\n7uvud684+4VVtWd8/qycfNDo/avI7u53dvd53b2Wk3/Wn+nuN60iu6rOGg8SzTgN/JtJVvJsnu7+\nmyQPV9WF41uXJFnJEwzWuSIuoy2DHbZDdpj9tf331+5lB8yiu5+oqrcm+WSSXUne1933rCq/qm5O\nsj/JC6rqWJJruvu9K4h+ZZI3J/nquE6eJH/U3R9fQfaLk9w4HuH/jCQf7O6VPu10Ii9KctvJfZ7d\nST7Q3Z9YYf7vJ7lp/MfzgSRvWVXwWKSXJvm9VWXuFFPusAn3V2KHrZr9tYL9tSWerg8AsBVslUtp\nAACTU4wAAAbFCABgUIwAAAbFCABgUIwAAAbFCABgUIwAAIb/F4TXVYh4gMANAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f4c2df79f98>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn import svm\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "clf = svm.SVC(gamma=0.001, C=100.)\n",
    "# training, let's us all the data but the last 4 instances\n",
    "clf.fit(digits.data[:-4], digits.target[:-4])\n",
    "# now predict the label for the last 2 instances\n",
    "print(\"Predicted Label:\", clf.predict(digits.data[-4:]))\n",
    "print(\"Actual Label:\", digits.target[-4:])\n",
    "\n",
    "plt.figure(figsize=(10,10))\n",
    "plt.subplot(221), plt.imshow(digits.data[-4].reshape(8,8), cmap='gray')\n",
    "plt.subplot(222), plt.imshow(digits.data[-3].reshape(8,8), cmap='gray')\n",
    "plt.subplot(223), plt.imshow(digits.data[-2].reshape(8,8), cmap='gray')\n",
    "plt.subplot(224), plt.imshow(digits.data[-1].reshape(8,8), cmap='gray')\n",
    "plt.title(\"Digits Image\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Scikit learn also lets you save your best model to disk for future uses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.externals import joblib\n",
    "joblib.dump(clf, 'filename.pk1')\n",
    "\n",
    "# load back the pickled model at a later time\n",
    "clf = joblib.load('filename.pk1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also update and fine-tune hyper-parameters after the model has been constructed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Linear Kernel:\n",
      "\tPredicted Labels: [5 4 8 8 4 9 0 8 9 8]\n",
      "\tActual Labels:    [5 4 8 8 4 9 0 8 9 8]\n",
      "\n",
      "Radial Basis Function Kernel:\n",
      "\tPredicted Labels: [5 4 5 5 4 9 5 5 5 5]\n",
      "\tActual Labels:    [5 4 8 8 4 9 0 8 9 8]\n"
     ]
    }
   ],
   "source": [
    "from sklearn import svm\n",
    "\n",
    "# classifer object\n",
    "clf = svm.SVC()\n",
    "# set hyper-parameters\n",
    "print(\"Linear Kernel:\")\n",
    "clf.set_params(kernel='linear').fit(digits.data[:-10], digits.target[:-10])\n",
    "print(\"\\tPredicted Labels:\", clf.predict(digits.data[-10:]))\n",
    "print(\"\\tActual Labels:   \", digits.target[-10:])\n",
    "print(\"\")\n",
    "\n",
    "print(\"Radial Basis Function Kernel:\")\n",
    "clf.set_params(kernel='rbf').fit(digits.data[:-10], digits.target[:-10])\n",
    "print(\"\\tPredicted Labels:\", clf.predict(digits.data[-10:]))\n",
    "print(\"\\tActual Labels:   \", digits.target[-10:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Scikit learn also supports multiclass and mutlilabel fitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 1, 1, 2])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "from sklearn.multiclass import OneVsRestClassifier\n",
    "from sklearn.preprocessing import LabelBinarizer\n",
    "\n",
    "X = [[1, 2], [2, 4], [4, 5], [3, 2], [3, 1]]\n",
    "y = [0, 0, 1, 1, 2]\n",
    "\n",
    "classif = OneVsRestClassifier(estimator = SVC(random_state=0))\n",
    "classif.fit(X,y).predict(X)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "In the above case, the classifier is fit on a 1d array of multiclass labels and the **predict()** method therefore provides corresponding multiclass predictions. It is also possible to fit upon a 2d array of binary label indicators:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 0, 0],\n",
       "       [1, 0, 0],\n",
       "       [0, 1, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = LabelBinarizer().fit_transform(y)\n",
    "classif.fit(X, y).predict(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, the classifier is **fit()** on a 2d binary representation of **y** using the **LabelBinarizer**. In this case **predict()** returns a 2d array representing the corresponding multilabel predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 1, 0, 0, 0],\n",
       "       [1, 0, 1, 0, 0],\n",
       "       [0, 1, 0, 1, 0],\n",
       "       [1, 0, 1, 0, 0],\n",
       "       [1, 0, 1, 0, 0]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import MultiLabelBinarizer\n",
    "\n",
    "y = [[0, 1], [0, 2], [1, 3], [0, 2, 3], [2, 4]]\n",
    "y = MultiLabelBinarizer().fit_transform(y)\n",
    "classif.fit(X, y).predict(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, the classifier is fit upon instances each assigned multiple labels. The **MultiLabelBinarizer** is used to binarize the 2d array of multilabels to **fit** upon. As a result, **predict()** returns a 2d array with multiple predicted labels for each instance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
